Monotonicity in Markov Reward and Decision Chains: Theory and Applications
نویسندگان
چکیده
منابع مشابه
Monotonicity in Markov Reward and Decision Chains: Theory and Applications
This paper focuses on monotonicity results for dynamic systems that take values in the natural numbers or in more-dimensional lattices. The results are mostly formulated in terms of controlled queueing systems, but there are also applications to maintenance systems, revenue management, and so forth. We concentrate on results that are obtained by inductively proving properties of the dynamic pro...
متن کاملCascade Markov Decision Processes: Theory and Applications
This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine the optimal performance and feedback controls, while some other cases are shown to lead to singular optimal control problems which are more difficult to solve....
متن کاملA Monotonicity in Reversible Markov Chains
In this paper we identify a monotonicity in all countable-state-space reversible Markov chains and examine several consequences of this structure. In particular, we show that the return times to every state in a reversible chain have a decreasing hazard rate on the subsequence of even times. This monotonicity is used to develop geometric convergence rate bounds for time-reversibleMarkov chains....
متن کاملMarkov Chains and Applications
In this paper I provide a quick overview of Stochastic processes and then quickly delve into a discussion of Markov Chains. There is some assumed knowledge of basic calculus, probability, and matrix theory. I build up Markov Chain theory towards a limit theorem. I prove the Fundamental Theorem of Markov Chains relating the stationary distribution to the limiting distribution. I then employ this...
متن کاملIncreasing Convex Monotone Markov Chains: Theory, Algorithm and Applications
We develop theoretical and algorithmic aspects of discrete-time Markov chain comparison with the increasing convex order. This order is based on the variability of the process and it is expected that one can get more accurate bounds with such an order although the monotonicity property is more complex. We give a characterization for finite state space to obtain an algebraic description which is...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Foundations and Trends® in Stochastic Systems
سال: 2006
ISSN: 1551-3092,1551-3106
DOI: 10.1561/0900000002